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Abstract 

Let  X  =  {X(t)  :  t  >  0}  be  a  positive  recurrent  synchronous  process  (PRS),  that  is, 
a  process  for  which  then  exists  an  increasing  sequence  of  random  times  r  =  {r(i)} 
such  that  for  each  k  the  distribution  of  9r(k)X  d=  {X(t  +  r{k))  :  t  >  0}  is  the  same 
and  the  cycle  lengths  Tn  =  r(n  +  1)  -  r(n)  have  finite  first  moment.  In  the  present 
paper  we  investigate  conditions  under  which  the  Cesaro  averaged  functionals  /!,(/)  d= 
\f^E(f(d,X))ds  converge  uniformly  (over  a  class  of  functions)  to  r(f),  when  r  is  the 
stationary  distribution  of  X.  We  show  that  ^(/)  — *  x(/)  uniformly  over  /  satisfying 
ll/IU  <  1  (total  variation  convergence).  We  also  show  that  to  obtain  uniform  convergence 
over  a 11  f  satisfying  J/j  <  j  (g  €  L*(r)  fixed )  requires  placing  further  conditions  on 
the  PRS.  This  is  in  sharp  contrast  to  both  classical  regenerative  processes  ind  discrete 
time  Harris  recurrent  .Mario r  chains  (when  renewal  theory  can  be  applied)  when  such 
uniform  convergence  holds  without  any  further  conditions.  For  continuous  time  positive 
Ferris  recurrent  Markov  processes  (when  renewal  theory  can  not  be  applied)  we  show  that 
these  further  conditions  an  in  fact  automatically  satisfied.  In  this  context,  applications 
to  queueing  models  an  given. 
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1.  Preliminaries  and  Introduction 

Throughout  this  paper,  X  =  {X(t)  :  t  >  0}  will  denote  a  stochastic  process  taking  values  in  a  complete 
separable  metric  state  space  5  and  having  paths  in  the  space  V  =  1?5[0,oc)  of  functions  /  :  R+  —  S  that 
are  right  continuous  and  have  left  hand  limits.  V  is  endowed  with  the  Skorohod  topology  (and  is  a  complete 
separable  metric  space).  (U,P,P)  will  denote  the  underlying  probability  space  and  we  view  X  as  a  random 
element  of  V.  Let  A  denote  an  arbitrary  fixed  element  not  in  the  set  S.  We  then  endow  S  d=  S  U  {A}  with 
the  one-point  compactification  topology. 

Definition  1.1.  X  is  said  to  be  a  synchronous  process  with  respect  to  the  random  times  0  <  r(0)  < 
r(l)  <  ■  ■  •  (  with  linVi—oo  r(n)  =  oc  a.s.  )  if  {Xn  :  n  >  1}  forms  a  stationary  sequence  in  the  space  Vf , 
where 

v  _  /  X(T{n-  l)  +  t),  ifO<t<Tn; 
n{)  -\x  ift>Tn. 

Tn  d=  r(n)  —  r(n  —  1)  is  called  the  n,h  cycle  length,  X„  is  called  the  n,h  cycle  and  we  refer  to  (r(n))  as  the 
synch-times  for  X  with  counting  process  N(t)  =  the  number  of  synch  times  that  fall  in  the  interval  [0,1]. 

Definition  1.2.  A  synchronous  process  X  is  called  non- delayed  if  r( 0)  —  0  a  s.;  delayed  otherwise.  It 
is  called  positive  recurrent  if  E(T\)  <  oo;  null  recurrent  otherwise.  It  is  called  ergodic  if  it  is  positive 
recurrent  and  the  invariant  <r-field,  1,  of  {.V,,,  Tn]  is  trivial.  A  d=f  gf’fyj  is  called  the  rate  of  the  synch  times. 
A  dTf  is  called  the  conditional  rate. 

From  now  on,  PRS  will  be  used  to  abbreviate  positive  recurrent  synchronous  process.  To  help  the 
reader,  an  appendix  is  included  at  the  end  of  this  paper  giving  a  brief  introduction  to  PRS's. 

9t  ;  V  — *  V  denotes  the  shift  operator  ( 6tx)(s )  =  x(t  +  s),  P°  denotes  the  probability  measure  under 
which  X  is  non-delayed;  P°(A)  =  PiQ^X  €  A)  and  P*  denotes  the  probability  measure  under  which  X 
has  the  stationary  distribution  ir  (see  the  Appendix). 

The  important  point  here  is  that  at  the  random  times  r(i),  X(t)  and  its  future  probabilistically  start 
over.  However,  in  contrast  to  classical  regenerative  processes  (CRP’s),  or  the  regenerative  structure  found 
in  Harris  recurrent  Markov  chains  (HRMC’s),  the  future  is  not  necessarily  independent  of  any  of  the  past 
(r(l), . . . ,  r(t);  X(s)  :  0  <  s  <  r(k)}.  In  particular  r  does  not  (in  general)  form  a  renewal  process 


and  hence  renewal  theory  does  not  apply  to  synchronous  processes.  Natural  questions  arise, 
however,  as  to  which  of  known  limit  theorems  etc.  that  hold  for  CRP’s  and  HRMC’s  actually  do  not  depend 
upon  renewal  theory  and  can  in  fact  be  extended  to  cover  PRS’s.  One  such  result  was  given  recently  in  [4] 
where  it  was  shown  that  for  a  PRS,  the  distributions  of  9,  o  X  d=f  {X(s  +  <):<>  0}  are  in  fact  tight  in  the 
function  space  D.  But  what  about  limit  theorems  for  PRS’s?  For  example,  although  N(t)/t  — ►  A  P  a.s. 
as  t  — •  oo,  what  can  be  said  about 

E(N(t))/t  — >  EX,  (1.1) 

which  does  hold  true  for  a  renewal  process?  Similarly,  for  an  ergodie  PRS,  although  ,  °X)ds  — » 

tt(/) ,  P°  a.s.  for  any  /  €  L i(t),  what  can  be  said  about 

?,(/)“=  l-fE°f{6,ox)ds  —  *U),  /€li(»),  (1.2) 

1  Jo 

or 

sup  |J?<(/)  —  ir(/)|  —  0,  (1.3) 

ll/ll— <i 

or  (more  generally) 

sup  |JT,(/)  -  ir(/)| — <0,  for  each  g  €  (1.4) 

\)\<i 

all  three  of  which  holds  true  for  CRP’s  and  HRMC’s? 

We  show  that  (1.3)  is  always  true  for  a  PRS  (ergodie  or  not),  whereas  (1.1),  (1.2)  and  (1.4)  require 
extra  conditions  (even  in  the  ergodie  case).  These  extra  conditions  turn  out  to  be  automatically  satisfied  for 
continuous  time  Harris  recurrent  Markov  processes  (HRMP’s).  In  this  context  we  give  some  applications  to 
queueing  models. 

2.  Limit  theorems  for  N(t) 

In  this  section  we  present  counterexamples  showing  that  (1.1)  is  false  in  general.  In  fact  we  show  that  even 
in  the  ergodie  case  it  is  possible  that  E°N(t )  =  oo. 

Let  r  =  (r(n)}  be  the  synch  times  of  a  non-delayed  PRS  X.  Let  N(t)  denote  the  corresponding  counting 
process.  Under  P°,  X  is  non-delayed  and  the  point  process  r  is  called  a  Palm  version  in  which  case  {T„} 
forms  a  stationary  sequence.  Under  P‘,  X  is  stationary  as  is  the  point  process  r  (see  for  example,  [7]). 
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Example  (1)  Let  Z  be  a  r.v.  such  that  P(Z  >  1)  =  1  and  E(Z)  =  oo.  Define  Tn  =  1  /Z  (n  >  1);  thus 
r(n)  =  n/Z  and  E(T„)  <  1  <  oo.  Observe  that  P(N(t)  >  n)  =  P(r(n)  <  t)  =  P(tZ  >  n)  so  that  indeed 
£(.V(t))  =  oo  for  all  t  >  0.  Observe,  however,  that  {T„}  is  not  ergodic;  its  invariant  (7-field  is  precisely  <r(Z). 

VVrhereas  Example  (1)  is  not  ergodic,  our  next  example  is. 

Example  (2)  Consider  a  discrete  time  renewal  process  with  cycle  length  distribution  V  =  {pk  ■ 
k  >  1}  having  finite  and  non-zero  first  moment,  1/p,  but  infinite  second  moment.  Let  D(n)  denote  the 
corresponding  discrete  time  forward  recurrence  time  process.  B  is  a  positive  recurrent  Markov  chain  with 
invariant  probability  distribution  an  d=f  P^2t>n  Pi  (tbe  equilibrium  distribution  of  P).  a  has  infinite  first 
moment;  kak  =  oo.  Let  h(k)  =  l/(Jt2  +  1)  and  define  a  point  process  by  Tn  =  h(B(n))\  r(n)  = 

T\  +  T2  + - 1 -Tn-  Observe  that  0  <  T„  <  1  for  all  n.  Under  a,  {Tn}  is  a  stationary  ergodic  sequence 

and  hence  corresponds  to  a  Palm  version  with  0  <  A  d=f  {£o/i(B(0))}-1  <  oo.  It  is  also  positive  recurrent 
regenerative;  it  regenerates  whenever  £(n)  =  0.  Let  7  =  min{n  >  0  ;  £(n)  =  0},  and  observe  that  7  =  3(0). 

Let  M  =  Consider  the  random  time  T  =  r(7)  =  T)  + - f  T-,.  Observe  that  T  <  M  <  o o  and 

that  N(T\Xq  =  k)  =  k.  Thus  for  t  >  M  we  obtain 

E°N(t)  =  f2Eo{N(t)\B(0)  =  k}ck 

ks  1 

>f^E°{N(T)\B(0)  =  k}Ok 
k  =  l 
00 

=  ***  =  OO. 

k  =  l 

The  important  point  here  is  that  in  general,  {N{t)/t  :  t  >  0}  is  not  uniformly  integrable  (UI).  We  do, 
however,  have  some  sufficient  conditions. 

Proposition  2.1.  Suppose  0  <  A-1  =  £°(T\)  <00  .If  either  there  exists  an  e  >  0  such  that  P°(Ti  >  ()  = 
1,  or  the  iate-eveat  times  {T„}  form  a  k-depeadent  process,  then  (1.1)  bolds. 

Proof:  Suppose  P°{T\  >  t)  =  1.  Then  for  all  t  >  0,  <  7  +  1,  Pa.s.  (even  in  the  delayed 

case)  and  hence  is  UI.  Suppose  now  that  the  T„  are  fc-dependent,  that  is,  for  each  n,  {T„+j  :  j  >  i}  is 
independent  of  {Tm  :  m  <  n}.  It  follows  that  for  each  i  (0  <  i  <  k  —  1)  T(i)  =  {Ttn+I  :  n  >  1}  defines 
a  (possibly  delayed)  renewal  process.  Let  N^{t)  denote  the  corresponding  t'*  counting  process.  Clearly 
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MO  £  .Vl0'(O  "f-  *vl>)(0  + - *-  M*  U(0-  Since  ;V  is  UI  for  each  i  (because,  for  each  i,  it  is  from 


renewal  process),  so  is  21il. 


Remark  (2.1):  By  changing  our  example  (2)  slightly,  we  can  actually  obtain  a  null  recur¬ 
rent  version:  Let  {T„}  and  N(t)  be  from  example  (2)  (under  a),  and  let  Hi  denote  the 
distribution  of  T„.  Let  {£*}  be  non-negative  i.i.d.  ~  Hi,  with  Hi  having  infinite  first 
moment.  Define  =  0,  7t+i  =  mm{n  >  :  B(n)  =  0).  Between  T^.i  and  Tyi,  insert 

Lh.  The  idea  here  is  to  start  off  the  kth  regenerative  cycle  with  and  then  proceed  as 
before.  This  gives  rise  to  a  new  sequence  of  interevent  times  Tn.  Taking  a  Palm  version 
of  this  new  point  process  yields  a  stationary  ergodic  sequence  7^  such  that  E(f%)  =  oc; 

~  (l—p)Hi  +pHi  where  p  —  1/(1  +  1/p).  Letting  N(t)  denote  the  associated  counting 
process,  we  obtain  £(iV(<))  >  (1  -p)E(N{t))  =  oo  for  t  >  M. 

Remark  (2.2):  In  our  example(2),  it  is  true,  however  (as  is  well  known  more  generally  in 
the  point  process  literature),  that  E'(N(t))  =  At  for  all  f  >  0  and  hence  that  the  intensity 
d—  £*(;V(1))  is  finite  and  is  equal  to  A.  It  is  only  the  Palm  version  that  can  blow  up. 


3.1  Uniform  limit  theorems  for  ,Y 

We  first  present  an  example  of  an  ergodic  PRS  together  with  an  /  €  Li(x),  such  that  pt(f)  =  oo.  In 
particular,  (1.2)  does  not  hold. 

Example  (3)  Consider  Tn  and  fl(n)  from  example  (2).  Form  a  semi-Markov  process  X(t)  by  using 
Tn  as  the  holding  time  for  B„.  Then  for  B(0)  «*  or,  X  is  an  ergodic  PRS  with  synch  times  r(n).  Now  choose 
an  /  >  0  such  that  fh  €  Li(a)  but 

21  /( 0M0  J2  a*  =  °°- 

•  t>» 

Then 

.  fT{1)  „  rw 

Jo  f(X(s))ds^J2^ij0  f(X(s))ds-,X(0)  =  k} 

=  ££°{/(*)M*);Mo)  =  t} 

k 

=*£ /(*)*<*>“*<  °°- 
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Hence  /  €  L\( it).  On  the  other  hand,  for  t  >  M, 

rt  ft 

*?,(/)  =  E°  /  f(X(s))ds  >  E°  I  f(X(s))ds 
Jo  Jo 


=  Te°{[T  f(X(s))ds;X(0)  =  k} 

k  Jo 

=  £  /(*  -  i)*(*  -  i);*(0)  =  *} 


*  j=o 

=  £^£/ow) 

*  J=0 

oo 

=  £  fU)Hi)  £  at  =  oo. 

;=o  *>> 

We  do,  however,  have  the  following 

Theorem  3.1.  If  X  is  PRS  and  g  €  Lf(x)  such  that  \E  f‘AT(0)  g(6,X)ds  — -  0  and  o  X)ds  : 

t  >  0}  is  UI  under  P°,  then 

sup  iJT,(/)  -  ir(/)|  — •  0.  (31) 

U\<s 

Before  proving  Theorem  3.1  we  state  an  important  corollary  obtained  immediately  by  using  the  function 
3—1- 


Corollary  3.1.  If  X  is  a  PRS  then  p,  converges  to  ir  in  total  variation. 

Proof  of  Theorem  3.1:  Assume  at  first  that  0  <  /  <  g  and  that  X  is  non-delayed.  For  f  >  0  let  A(e,t) 
denote  the  event  {JV(t)  >  (A  +  e)t}.  Let  Jn  =  Jn(f )  =  Sr^-x)  /(*«  °  X)di-  Let  denote  E°  conditional 
on  the  invariant  e-field  I.  Then 

~Ei  j'  }{6,X)ds  =  jEjiJ'  f(B,X)ds;A(e,t)c}  +\El{J  W*X)dr,A(t,t)} 

<  +  \Ez{  J*  g(9,X)ds;A(t,  t) }  (3-2) 

<  XEjJi  +  (f  +  - )EiJi  +  ~Ej{  j  g(9,X)ds;A(c,t)}- 

t  *  Jo 

Taking  expectations  in  (3.2)  with  respect  to  E°  yields 

jE°  £  f{9,X)ds<x(f)  +  ((+  j)E°Ji  +  \e°{Jo  g{0,X)ds\A{t,t)}-  (3-3) 

By  the  uniform  integrability  hypothesis,  the  last  term  in  (3.3)  tends  to  zero.  Moreover,  e  was  arbitrary.  We 
thus  obtain 

Km  sup{J7,(/)  -  *(/)}  <  0.  (3-4) 
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In  a  similar  manner  we  obtain  a  lower  bound:  For  e  >  0 

r(i-«)0 


1  r‘  1  1 

7 EX  f{9,X)ds  >  -Ei{  Y 
1  Jo  1  iTi 

( \  —  At  i  i  r(*—oo 

>(- - f-)ExJ1-\EI{  Y  Jk(gy,A(-c,tY} 


i=l 

f(A-0«1 


t  '  t 

>  \EIJl~(C+  l-)Ez Ji-jEz {'  Y  M9)-,A(-€,t)e}, 

1  z  t=i 

which  after  taking  expectations  yields 

1  1  fi!l 

?«(/)  -  *(/)  >  -(«  +  (3.5) 

fc  =  l 

Since  g  6  Li(t),  j  Hii'i  -/if?)  is  UI  since  it  converges  a.s.  to  XEjJi(g)  and  has  mean,  £’0{-^.£’z./i(?)}, 
for  each  t.  Thus  the  last  term  in  (3.5)  tends  to  zero.  Consequently 

(3.6) 


lim  sup{x(/)-JT,(/)}  <  0, 


and  we  thus  obtain  (3.1).  The  case  of  /  with  arbitrary  sign  can  be  handled  similarly;  we  leave  out  the  details. 

In  the  delayed  case,  we  have  on  the  one  hand  that 

1  r*  1  i  r* 

-E  f(6,oX)ds<-E  /  g(6,X)ds  +  -£°  f  {6,  oX)ds.  (3.7) 

t  Jo  *  J o  t  J0 

The  first  term  on  the  rhs  tends  to  zero  by  assumption,  thus,  giving  the  necessary  upper  bound.  On  the  other 
hand,  for  t  >  M  >  0 

E  ff{9,  o  X)ds  =  E{  f  f  {9,  o  X )d»\  r(0)  <  M}  +  Ec{  [  f(9,  c  X)ds ;  r(0)  >  M) 

J o  Jo  Jo 

>E{[  f{9,  o  X)ds\  r(0)  <  M) 

J  r(0) 

ft-M 

>E°{  f{9,  o  X)ds;  r(0)  <  M}  (3.8) 

Jo 

>  E°{  f  f{9,  o  X)ds}  -  E°{  fg{6,  o  X)ds;  r{ 0)  >  M} 

J  0  Jo 

>E0{[tf{9loX)ds}-E0{[  g{6,oX)ds}-  E°{  [  g{9,  o  X)ds;r(0)  >  M}. 

Jo  Jt-M  Jo 

Using  in  M  =  ct,  0  <  t  <  1,  in  (3.8)  yields 

jEf'fiB,  o  X)ds  >  J. E°{J*f{6 ,  o  X)d3 }  -  \&{j'  g{9.  o  X)ds}  -  ±E°{j‘{g(0.  o  X)ds;  r(  0)  >  rf}}. 

(3.9) 

The  last  integral  above  tends  to  zero  by  the  UI  assumption  under  P°.  The  middle  integral  converges  to 
tE°{\EzJ\{g)}.  Thus,  letting  c  tend  to  zero  yields  (together  with  (3.7))  the  desired  result.  ■ 
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Proposition  3.1.  For  a  PRS,  if  either  there  exists  an  e  >  0  such  that  P°(T1  >  e)  =  1,  or  the  cyc.es  {Xn} 
form  a  /k -dependent  process,  then  (3.1)  holds  for  all  g  g  L+(tc)  su ch  that  E  /0Tl0)  g(9,X)ds  <  oo. 


Proof:  From  Theorem  3.1,  it  suffices  to  show  that  {} /0'p(S,  °  X)da  :  t  >  0}  is  uniformly  integrable  under 

P° .  If  P°(T\  >  f)  =  1  then  gf^gi^,  0  X)ds  <  |  Jk(g)  which  is  UI  for  g  g  Now  suppose  that 

the  cycles  are  fc-dependent  (in  particular,  X  is  ergodie).  Then 
„  *(<)+* 


[,g(d,oX)ds<  Ji(9) 

J°  j  =  i 


=  YJ^)  +  Y,  W9)iwt)>j-k). 

1=1  ;=i  +  l 

Bv  the  assumption  of  k-dependency,  the  indicator  >  j  —  k)  is  independent  of  Jj(g)  and  hence  taking 


expectations  in  (3.6)  yields 


E°  [  g(B,  o  X)ds  <  ( E°N(t )  +  *)£°  Jx($). 

Jo 


By  Proposition  0  1.  {,/V(t)/t}  is  UI  and  hence  by  (3.7)  so  is  {\Jog{B$  o  X)ds  :  t  >  0}. 


Remark f 3.1 ):  If  X  is  null  recurrent  and  non-ergodic,  it  is  still  possible  that  tr  as  defined  ir. 

(A. 3)  is  a  probability  measure.  In  this  case  Theorem  3.1  remains  valid.  Take  for  example, 
a  mixture  of  Poisson  processes:  Choose  a  r.v.  Y  such  that  P( 0  <  Y  <  1)  =  1  and 
E(\/Y)  =  oo.  Given  Y,  let  {-r(Jb)}  be  a  (non-delayed)  Poisson  process  at  rate  Y.  Define 
X(t)  as  the  forward  recurrence  time  of  this  point  process.  Then  the  invariant  <7-field  is 
precisely  <t(Y),  £(ri|y)  =  1/Y  and  hence  E{T\)  =  oo.  Moreover,  given  Y ,  the  (marginal) 
steady-state  distribution  of  A’(<)  is  exponential  at  rate  Y .  Thus  the  (unconditional)  steady- 
state  distribution  is  given  by  F{z)  =  1  —  E(e~Yt). 

Remark(3.2):  The  condition  j £/0<Ar^0)  g(9,X)ds  — >  0  iseqivaient  to  Ulof  {  j  /0<Ar(Q)  g{6,X)ds}. 


4.  Continuous  time  Harris  recurrent  Markov  processes 

In  this  section  we  establish  uniform  limit  theorems  for  continuous  time  Harris  recurrent  Markov  processes 
(HRMP’s)  analogous  to  those  already  known  (in  the  literature)  to  be  true  for  discrete  time  Harris  recurrent 
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Markov  processes,  called  Harris  recurrent  Markov  chains  (HRMC’sJ.  Although  renewal  theory  can  be  used 
to  analyze  HRMC's,  the  same  is  not  true  for  HRMP’s  (as  defined  below). 

Let  (Z(t)  :  t  >  0}  denote  a  Markov  process  with  Polish  state  space  S  and  paths  in  T>s ■  We  shall  always 
assume  that  Z  has  the  strong  Markov  property. 

Z  is  called  Harris  Recurrent  if  there  exists  a  non-trivial  cr-finite  measure  p  on  the  Borel  sets  of  S  such 
that  for  any  Borel  set  A  C  S 

p(A)  >  0  =>  P,  (/  1a  o  Z(t)dt  =  oo'j  =  1  for  all  z.  (4.1) 

It  is  known  that  a  HRMP  has  a  unique  invariant  measure  (up  to  multiplicative  constant);  see  for 
example,  [2]  and  [10].  If  the  invariant  measure  is  finite  then  it  is  normalized  to  a  probability  measure  in 
which  case  Z  is  called  positive  recurrent.  In  Theorem  2  of  [10],  it  is  proved  that  a  Markov  process  Z  is  a 
positive  HRMP  if  and  only  if  it  is  a  positive  recurrent  one-dependent  regenerative  (od-R)  process,  that  is. 
an  ergodic  synchronous  process  with  one  dependent  cycles.  In  particular,  Corollary  3.1  and  Proposition  3.1 
both  apply  to  positive  HRMP’s.  So,  for  example,  given  any  initial  state  Z o  —  z,  it  follows  that  the  Cesaro 
averaged  measures  JZJ(A)  d=  \JqE,Ia  o(6,Z)ds  converge  to  ir  in  total  variation  as  t  — •  oc. 

Once  the  od-R  points  have  been  selected  for  a  HRMP,  a  natural  question  arises  as  to  wether  or  not,  by 
placing  some  regularity  conditions  (non-lattice  (or  spread-out)  cycle  length  distribution,  etc.)  on  the  cycles 
of  an  HRMP  Z,  the  unaveraged  distributions  will  converge  weakly  (or,  even  better,  in  total  variation)  to  t, 
that  is.  if  p\{f)  Et(f(9,Z)}  — •  t(/)  for  ail  bounded  continuous  /.  The  answer  is  no;  a  counterexample 

is  given  in  Remark(3.2)  of  [10].  Also  see  example(l)  of  [4],  (It  is  true,  however,  that  for  each  z,  {/i[  :  t  >  0} 
is  a  tight  collection  of  measures  (see  Theorem  2.1  of  [4])). 

Continuing  in  the  spirit  of  Cesaro  convergence  we  have 

Proposition  4.1.  If  Z  is  a  positive  HRMP  with  stationary  distribution  ir  then  for  each  g  6  Lf{i r), 


sup  jJTj (/)  -  t(/)|  — ►  0  for  almost  every  z  w.r.t.  w.  (4.1) 

l/l<y 

Proof:  Let  r(z)  d=  E,  g(9,X)ds  and  Z  d=  {z  ;  r(z)  <  oo}.  From  Proposition  3.1  it  suffices  to  show 

that  ir(S)  —  1.  Now, 


ir{£)  =  XE°  I{r{Z{$)  <  oo )ds. 
Jo 


(4.2) 
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Moreover. 


/•'(D 

£°{r(Z(s);r(l)  >  ,}  =  E°{E°Z{,){  /  (s(^Z)du}; r(  1)  >  a)} 

Jo 

rH  x) 

=  £°{/  (g(9uZ)du  ;r(l)>s)} 

/■*•(  i) 

<  E°{J  (g(9uZ)du)  <  oo 

Thus  r(Z(s)I(r(\)  >  sj)  <  oc  P°  a.s.  and  hence  P°{r(Z(s))  <  oc;r(l)  >  s}  =  1  for  ail  s  >  0-  Integrating 
over  s  yields  the  result.  ■ 


Proposition  4.2.  If  Z  is  a  positive  BRMP  with  stationary  distribution  ir  then 


sup  (/)  -  ir(/)|  - 0. 

I/I<» 


for  all  g  6  L*(;r)  such  that  £°{/0ra)  u  \9uX)du)  <  oc. 


(4.3) 


Proof:  We  must  show  that  the  error  bounds  for  RIJ(/)  —  can  be  integrated  over  ;  with  respect  to 

t  .  From  the  bounds  obtained  in  (3.5)-(3.9)  it  suffices  to  show  that  h(z)  d=  E,  /0T(0>  g(9,X)ds  is  in  Li(:r). 
An  easy  calculation  yields 


r  rr  O)  /•r(i) 

J  *(dz)h(z)  =  A£°  J  9,o  J  g(9uX)duds 


=  a  r° 

=  A£° 


r 


*0)  yr(l) 


/ 


g(9uX)duds 

/"■(I) 

/  ug(9uX)du 

Jo 


Rcmark(4.1):  In  the  proof  of  Proposition  4.1,  the  assertion  that  ir(£)  =  1  amounts,  in 
the  terminology  of  discrete  time  Markov  chain  theory,  to  showing  that  a.e.  state  z  (with 
respect  to  t)  is  g  —  regular  (see  Proposition  5.13  of  Nummelin[6]).  In  fact,  Proposition 
4.1  cam  be  viewed  as  a  continuous  time  Cesaro-  average  analog  of  Corollary  6.7i)  in  [6]- 

Remark(4.2):  In  Asmussen  [1]  the  definition  of  HRMP  is  different  than  ours.  Ours  comes 
from  Azema,  Duflo  and  Revuz  [2].  Asmussen ’s  definition  is  more  restrictive  and  in  partic¬ 
ular  implies  the  existence  of  an  embedded  renewal  process. 
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5.  Applications  to  queues 


In  [9]  the  stability  of  open  Jackson  queueing  networks  is  established  where  service  times  are  i.i.d.  with  general 
distribution,  exogenous  interarrival  times  axe  i.i.d.  with  general  distribution,  and  the  routing  is  Markovian. 
We  present  here  some  immediate  consequences  of  section  4  in  the  context  of  the  above  stability  result. 

Consider  a  c  node  queueing  network  with  the  n,h  exogenous  customer  (denoted  by  Cn)  arriving  at  time 
tn  with  0  <  f  i  <  t2  <"  •  •  •  and  limrl_00  tn  =  oo.  Each  node  is  a  FIFO  single  server  station  (with  unlimited 
size  waiting  room).  Upon  arrival,  each  customer  is  assigned  (independent  of  t.ie  past)  an  initial  station 
according  to  the  initial  distribution  V  =  (pi, Pi,. ••,?«).  Routing  is  Markovian  :  after  completing  service 
at  node  i,  a  customer  is  routed  (independent  of  all  else)  to  the  end  of  the  queue  at  node  j  with  probability 
r,  j ,( 1  <  i.j  <  c).  In  addition,  w->  let  r,  3  denote  the  probability  of  leaving  the  system  after  a  service 
completion  at  node  i  (and  going  borne).  Thus  each  customer’s  sequence  cf  routings  forms  a  Markov  chain 
(with  initial  distribution  V)  which  we  assume  has  precisely  one  set  of  absorbing  states,  the  singleton  {g}. 
R  =  (rtJ)  is  called  the  routing  matrix.  Service  times  at  the  i,h  node  {£*(»)  :  k  >  0}  are  handed  out  by  the 
server  and  assumed  i.i.d.  with  distribution  Gi  and  mean  0  <  1/p*  <  oo;  S(«)  will  denote  a  generic  service 
time  ~  G,.  We  assume  that  exogenous  arrival  epochs  (t„)  form  a  renewal  process  with  rate  0  <  A  <  oo 
and  (i.i.d  )  interarrival  times  T„  =  tn+ 1  —  tn;  E(T„)  =  1/A.  T  will  denote  a  generic  interarrival  time.  We 
let  In  denote  the  initial  node  for  C„;  (f„)  forms  an  i.i.d.  sequence  with  distribution  V .  The  service  time 
sequences,  the  interarrival  time  sequence,  and  the  initial  node  sequence  are  assumed  independent.  This  is 
called  a  Jackson  open  network  (JON)  with  general  distribution  i.i.d.  input. 

Let  Qn  =  (Q„(l),  Qn(2), . . . ,  CJn(c))  denote  the  queue  lengths  (not  including  those  in  service)  at  the 
c  nodes  at  time  t„—  and  Y„  =  (Yn(l),  Y„(2), . . . ,  Yn(c))  the  residual  service  times  (set  to  zero  if  server  is 
free).  It  easily  follows  (by  all  the  i.i.d.  assumptions)  that  the  process  Xn  =  (Qn,Yn)  forms  a  Markov  chain 
with  state  space  X  —  N+  x  R?+.  Let  X(t)  =  ( Q(t),Y(t ))  denote  the  queue  length  vector  and  residual 
service  time  vector  at  time  t ;  X„  —  X(tn—).  Let  B(t)  denote  the  forward  recurrence  time  of  the  renewal 
process  of  exogenous  arrivals;  B(t)  is  the  time  until  the  next  arrival  after  time  t.  It  is  easily  seen  that 
Z(t)  =  (A(t),  £?(<))  is  a  Markov  process  with  state  space  5  =  X  x  R+  and  paths  in  V. 

For  each  i  (1  <  i  <  c),  let  Nn(i)  denote  the  (random)  number  of  times  that  C„  will  desire  service  at 


11 


node  i.  Mi  =f  £(>'„(»))  <  oo  (in  fact  has  finite  moments  of  all  orders).  Define  A<  d=  XMi,  we  refer  to  Xt  as 
the  total  arrival  rate  to  n ode  i.  It  is  well  known  that  the  Aj’s  are  the  unique  solution  to  the  following  set  of 
c  equations  : 


A,  =  Ap,  +YlX:rii  +  A*rii>  !<»<c.  (2.1) 

Finally,  we  define  p,  d=  A it  represents  the  long  run  average  rate  at  which  work  arrives  exogenously 
to  the  system  destined  for  node  «;  p  d=  p\  +  •  •  •  +  pe  denotes  the  total  long  run  average  rate  at  which  work 
arrives  exogenously  to  the  system. 

The  following  two  theorems  are  proved  in  [9]: 

Theorem  5.1.  The  Markov  chain  X  =  {.Yn}  for  JON  is  Harris  ergodic  if  pi  <  1  for  each  i  (1  <  i  <  c). 

Theorem  5.2  If  A  >  0  (i.e.  interarrival  times  have  finite  first  moment )  then  the  Markov  process  Z  for 
JOS  is  a  positive  Harris  recurrent  Markov  process  (HRMP)  if  pi  <  1  for  each  i  (1  <  *  <  c).  In  particular  it  is 
positive  recurrent  one-dependent  regenerative  (od-R)  with  a  unique  steady-state  distribution  tt.  Moreover, 
Z(t)  converges  to  ir  in  total  variation  if  and  only  if  A  is  spread-out.  (In  general,  ir{(X(t)  =  0}  =  0  and 
hence  the  regeneration  points  of  Z  are  not  described  by  consecutive  visits  of  X  to  the  empty  state). 

From  the  above  theorem  we  see  that  Z  is  an  ergodic  PRS  with  one-dependent  cycles.  Thus,  so  is  any 
continuous  functional  f(Z(t))  such  as  total  queue  length  Qr(t)  (sum  of  the  c  queue  lengths).  Moreover, 
total  work  in  system  u i(t)  is  also;  w(t)  denotes  the  sum  of  all  remaining  service  times  of  all  customers  in  the 
system  (including  their  feedback)  at  time  t  (see  section  4  of  [9]).  We  thus  obtain  the  following  special  cases 
of  the  results  in  section  4: 

Proposition  5.1.  For  a  JON  with  pi  <  1  for  each  i  (1  <  «  <  c),  the  following  hold: 

P,(w(s)  €  )ds  — >  Pt(ui(Q)  €  ■)  in  total  variation  for  each  z. 

P,(Qt(s)  €  -)ds  — •  Pt(Qt(0)  €  •)  in  total  variation  for  each  z. 

If  ^(ti^O))  <  oo  then 

E,(w(s))ds  — •  £,(ui(0))  for  almost  every  :  w.r.t.jr. 
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-  I  £,(Qr(s))ds  — —  Et(Qt( 0))  for  almost  every  r  w.r.t.ir. 
t  Jo 

APPENDIX  :  A  brief  introduction  to  synchronous  processes 

Our  use  of  the  word  synchronous  is  from  [3].  Other  names  have  been  given  to  a  synchronous  process; 
for  example  Serfozo  [8]  refers  to  them  as  semi-stationary  processes.  In  Rolski[7]  they  arise  as  Palm  versions 
of  stationary  processes  (associated  with  point  processes).  Closely  related  to  this  is  the  general  theory  of 
stationary  marked  point  processes.  In  any  case,  the  ergodic  properties  of  synchronous  processes  are  well 
known  in  the  literature.  We  state  several  such  results  the  proofs  of  which  can  be  found  in,  for  example  [3], 
[4],  [5],  [7]  and  [8]. 

Let  9t  :  V  —  V  denote  the  shift  operator  ( 9,x)(s )  =  x(t  -f  s). 

Theorem  A.l.  Suppose  X  is  a  PRS  and  f  :  Vs  —  3i  is  measurable.  Let  Jn  =  Jn(f)  d=  f(9,oX)dt. 

If  M\f\)  <  oo  a.s.  and  if  either  /  >  0  a.s.  or  E{Ji{\f\)}  <  oc  then 

,5s  •>*)■'•=  H£fj-  - 

where  I  denotes  the  invariant  <r-field  associated  with  {(.Y„,Tn)}. 

Let  P°  denote  the  probability  measure  under  which  X  is  non-delayed,  that  is,  P°(X  €  A)  =  P{9T(i)° X  6 

A). 

Corollary  A.l.  Under  the  conditions  of  Theorem  A.l,  if  in  addition  I  is  trivial  ( every  set  has  probability 
0  or  1)  then  {Jn,T„  :  n  >  1}  is  ergodic  and  hence  a.s. 

lira  \  f  f(0,  o X)ds  =  =  A  ^  P°(0,  oXtA-, r(l)  >  s)ds.  (A.l) 

'-“Wo  Jo 

Under  these  circumstances,  X  is  called  ergodic. 

The  following  Corollary  follows  from  (A.l)  by  an  elementary  application  of  Fubini’s  Theorem  and  the 
Bounded  Convergence  Theorem. 

Corollary  A. 2.  Under  the  hypothesis  of  Theorem  A.l,  if  in  addition  f  is  bounded  then 

*(/)  *  i  ['EM. .  .V)*  -  .(/)  *  £  {  fj£El}  .  M 
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it  above  defines  a  measure  on  V  and  (for  reasons  given  below  in  Proposition  A.l)  is  called  the  stationary 
probability  measure  for  X .  In  particular,  by  choosing  /  =  1^  (an  indicator  function),  we  have  J1,(A)  — 
for  each  Borel  set  A  off);  thus  the  Cesaro  averaged  distributions  converge  weakly. 


Proposition  A.l.  Let  it  be  the  stationary  measure  of  a  PRS  X.  Then  under  r,  9  —  (9,)  is  measure 
preserving  on  V,  that  is,  for  each  Borel  set  A,  *(A)  =  ir(9-,A)  for  all  s  >  0.  In  particular,  if  X  has 
distribution  it,  then  X  is  time  stationary,  that  is,  9xX  has  the  same  distribution  for  each  t  >  0. 


Let  P'  denote  the  probability  measure  under  which  X  has  distribution  ir,  that  is,  Pm(X  £  A)  =  ir(.<4). 
From  (A. 2)  we  obtain  for  an  ergodic  synchronous  process  that 


P'(X  £  A)  =  A  [  P°{9,  o  X  £  .4;  r(l)  >  s)ds. 
Jo 


(.4.4) 


If  X  is  positive  recurrent  but  not  ergodic  then  the  RHS  of  (A. 4)  still  defines  a  probability  measure  on 
V  (but  not  necessarily  the  same  as  the  it  from  (A. 3)).  In  fact,  more  can  be  said: 


Proposition  A. 2  For  a  PRS  tie  RES  of  (A.4)  defines  a  probability  measure  on  V  (in  general,  not  the 
same  as  r)  under  which  6  —  (9,)  is  measure  preserving. 
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